This invention relates to data processing, and more particularly, to a method and system for formatting data files that are subjected to a high volume of changes.
In many applications, there is a need for formatting data files that are subjected to a high volume of changes. One such application is in the banking industry wherein financial data and transactions from many banks or other financial institutions are transmitted to a data processing service provider for processing and the data processing service provider returns to the banks and financial institutions reports and files relating to the financial data that has been processed. Generally, financial data to be processed is provided by a very large number of banks, with typically hundreds of banks providing account transaction information to be processed on a daily basis. Thus, the data processing service provider accepts files from a large number of sources of data and distribute files to a large number of targets.
Formatted data files typically are used in the transmission of financial data, with the structure of the formatted files being agreed upon in advance by parties to the processing configuration. The maximum length of the file records that can be specified is a function of transmission protocol used, and, as an example, may be limited to no more than about two hundred fifty bytes.
However, change is inevitable. When transaction files are subject to rapid and high volumes of change, the task of announcing the change to all of the client banks and having the corresponding changes implemented simultaneously by all of the client banks is, for all practical purposes, impossible. On the one hand, if one client bank of a large number of client banks desires a change in format for one or more records, it can be difficult, if not impossible to convince other client banks to make that change. Even if all of the client banks were willing to make the change, it would be difficult, and essentially impossible, to get all of the client banks to change over to the new format at the same time. Moreover, most client banks do not want to make changes in format for the sole purpose of accommodating features pertaining to one or more other client banks.
There are several other options for accommodating increase in the size of records. For example, parallel files can be created, one with the old file format and a second with the new format. Another option is to create a second file to contain the new or expanded fields.
In most known prior art transaction interface systems, changes in format of a record are accommodated by expanding the size of the record. However, eventually, as the changed records increase in size, the maximum size supported by the transmission protocol is reached, thus limiting the changes that can be made.